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ABSTRACT 

Objective: To assess the inclusion of patients as 
international research partners in Outcome Measures in 
Rheumatology (OMERACT) conferences and how this 
has influenced the scope and conduct of outcomes 
research in rheumatology. 

Design: A thematic content analysis of OMERACT 
internal documents, publications and conference 
proceedings, followed by a responsive evaluation 
including 32 qualitative semistructured interviews. 
Setting: The international, biannual research 
conference OMERACT 10 (Malaysia, 2010). 
Participants: Senior researchers (n=10), junior 
researchers (n=2), representatives of the 
pharmaceutical industry and regulators (n=2), 
conference staff (n=2), new patient delegates (n=8) 
and experienced patient delegates (n=8). 
Results: The role of patients evolved over 10 years 
from a single patient focus group to full 
participation in all areas of the meeting and 
inclusion in research group meetings between 
conferences. Five main categories of impact 
emerged: widening the research agenda; including 
patient relevant outcomes in core sets; enhancing 
patient reported instruments; changing the culture of 
OMERACT and consequences outside OMERACT. 
Patient participants identified previously neglected 
outcome domains such as fatigue, sleep 
disturbances and flares which prompted collaborative 
working on new programmes of research. Specific 
benefits and challenges for patients and 
professionals were identified, such as personal 
fulfilment, widening of research interests, difficulties 
in establishing equal partnerships and concerns 
about loss of research rigour. 
Conclusions: Including patients as partners in 
OMERACT conferences has widened its focus and 
adjusted the way of working. It has resulted in new 
developments in the research agenda and the use of 
more patient-relevant outcomes in clinical trials. 
These collaborations have influenced perceptions and 
beliefs among many patients and researchers, and 
led to wider patient involvement as partners in 
research. 



ARTICLE SUMMARY 



Article focus 

■ Since 2002, patients have participated as collab- 
orative partners in the biannual conference on 
Outcome Measures in Rheumatology (OMERACT) 

■ Although the contribution of patients has been 
praised and there is a widespread call for scien 
tific publications on the impact of engaging with 
patients, no systematically obtained evidence has 
been published to support the idea that the 
structural involvement of patients in research 
conferences is beneficial. 

■ Our qualitative study reports the combined 
results of a thematic document analysis and 32 
semistructured interviews with all stakeholders 
including researchers, patient participants and 
representatives from the pharmaceutical industry 
and international regulators. 

Key messages 

■ Long-term engagement with arthritis patients in 
OMERACT conferences has significantly influ- 
enced outcome research in the field of 
rheumatology. 

■ Patients have successfully contributed to the 
research agenda of OMERACT by identifying new 
domains that are important for patients, and pro- 
vided the patient perspective in the development 
of core outcome measurement sets and the 
development of patient-reported outcome 
measures. 

Strengths and limitations of this study 

■ Triangulation of the combined review of the 
document analysis and interviews, together with 
the active involvement of all team members 
representing a variety of perspectives in the 
phase of data analysis and interpretation, has 
enhanced the validity of the study. 

■ The unique context of the OMERACT conferences 
may limit the generalisability of the results, so 
comparable evaluation studies in other confer- 
ence formats would be worthwhile. 
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Incorporating the patient perspective in healthcare 
research is strongly promoted by policy makers, 1-4 
funding bodies and international regulators. Many the- 
oretical benefits from patient involvement in research 
have been reported, 5-8 such as improving the relevance 
of research questions, improving the recruitment of 
study participants, and increasing the chances for 
funding and dissemination of results. In addition, there 
is an increasing recognition of the essential role of 
patients in outcome research. 9 The USA Food and Drug 
Administration has made patient involvement mandatory 
in the process of the development of patient-reported 
outcome (PRO) measures, 10 11 and in the context of 
Core Outcome Measures in Effectiveness Trials 
(COMET), patient contributions are seen as crucial in 
defining domains that are relevant to include in core 
outcome measurement sets for clinical trials. 12 
Development of core outcome sets might lead to less 
variety of incomparable and inappropriate outcome 
measures, more patient-oriented endpoints and less bias 
by selective reporting of only positive or statistically rele- 
vant outcomes. 13 Core outcome measurement sets may 
ease the work of systematic reviewers in synthesising the 
results of multiple studies. 14 15 The question is, however, 
whether these theoretical benefits of patient involve- 
ment in outcome research make any difference in 
practice. 

The international group Outcome Measures in 
Rheumatology (OMERACT), which defines core 
outcome measurement sets in rheumatic diseases, first 
included patient participants at its sixth biannual confer- 
ence in 2002 and has continued to do so. This provides 
an opportunity to analyse the consequences and address 
the important question of whether patient participation 
has resulted in any demonstrable impact on the nature 
of its research activity. 

Patient involvement in OMERACT has been presented 
as beneficial and the 2002 conference report concluded 
that 'the preliminary success of this forum' was the basis 
for 'continued and possibly expanded patient participa- 
tion at the next OMERACT meeting'. 16 Two conferences 
later, others perceived the involvement of patients as 
"indicative of the beginning of a paradigm shift in think- 
ing about RA outcomes over the last 5 years." 17 Since 
then, OMERACT has formulated three principles recog- 
nising the essential role of patients in outcome 
research. 18 First, patients' input is indispensable when 
defining relevant outcome measures, identifying 
domains that are important from the perspective of 
patients, and assessing the feasibility of measurement 
tools. Second, structural involvement of patients during 
the whole research process provides face validity. Third, 
OMERACT intends "to ground theoretical discussions in 
the lived experience of arthritis, and in concepts which 
can be readily communicated to patients to help with 
therapeutic decision making." 18 

However, the validity of these arguments has never 
been substantiated by robust evidence for the 



effectiveness of patient participation in OMERACT, and 
it is not clear whether or how this involvement has influ- 
enced methodologies, procedures, attitudes and 
research outcomes. Therefore, the objective of this study 
is to describe and evaluate the contributions made by 
patients since OMERACT started implementing struc- 
tural patient participation in its conferences. We review 
the impact of patients on the research agenda and the 
development of PROs and explore how including 
patients has influenced the culture and structure of the 
OMERACT conference. 



METHOD 

Patient participation in research is a new phenomenon 
that is often not reported or reflected on in scientific 
publications. This lack of written sources in the scientific 
literature complicates the study of the process and 
impact of patient participation through a review of the 
relevant literature. A provisory search using PubMed 
(March 2010) for the terms 'patient participation' OR 
'patient involvement' OR 'user involvement' OR 'con- 
sumer involvement' AND 'OMERACT' did not generate 
any relevant reference. Therefore, we conducted a 
content analysis of relevant documents (any written 
material on the topic of patient participation) . 

Documents are a stable, rich source of contextual 
information, providing well-grounded data on events or 
situations at low costs. A sound document analysis is 
rule-bound, systematic, following a coding process where 
raw data are aggregated into units describing the 
content. 19 We included OMERACT conference proceed- 
ings as published by The Journal of Rheumatology (1992- 
2010) and 'grey literature' such as correspondence, invi- 
tations, session reports, emails and OMERACT policy 
documents. The review focused on the arguments, 
reception and evolution of patient involvement in 
OMERACT conferences and the contributions made by 
patients. 

Subsequently, a responsive evaluation took place 
during OMERACT 10 (Malaysia, 2010) using qualitative 
interviews with representatives of stakeholders. 
Responsive evaluation is grounded in the hermeneutic 
research tradition and is used by social scientists to inter- 
pret meanings that participants attribute to a phenom- 
enon, here the history and impact of a decade of patient 
participation from the perspective of the conference 
delegates. It samples all stakeholders and does not seek 
consensus, but respects the plurality of opinions, values 
and interests. This ensures that no perspective is omitted 
as the result of an imbalance of power. 20 

The first author (MW) has been involved in 
OMERACT since 2002 as a patient participant. He has a 
rheumatic condition and has been educated as a respon- 
sive researcher. Characteristics for a responsive 
researcher are a multiple partiality and the intent to 
enhance mutual understanding among all stakeholders. 
The last author (JK) has been involved in OMERACT 
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since the first conference (1992) and has been the 
leader of the patient perspective workshop between 
2002 and 2012. Having witnessed the involvement of 
patients first-hand from the very beginning, MW and JK 
provided useful information to start the research; yet it 
also alerted them to critically reflect how this engage- 
ment influenced the research, and how to prevent bias. 
Therefore, two independent experts (TA, MK) were 
added to the team. They had no relations with the 
OMERACT conference and its participants, and TA 
acted as a peer-debriefer discussing dilemmas and chal- 
lenging methodological decisions with MW. 21 

The first author held 32 semistructured interviews 
before, during and after the 10th conference (table 1) 
and included senior (n=10) and junior researchers 
(n=2), representatives of the pharmacological industry 
and regulators (n=2), conference staff (n=2), new 
patient participants (n=8) and experienced patient parti- 
cipants (n=8). The interviewees were invited and 
informed by email. The patient participants were aware 
of the purpose of the study through a one-page 
announcement in the preconference patient pack and 
were asked for informed consent. In the Netherlands, 
no ethical approval is required for non-intrusive inter- 
views only 

Twenty-eight interviews were recorded, transcribed by 
an independent secretariat and subjected to a responder 
check. Three interviews were summarised in a report, 
one interview took place without protocol and, on 
request of the interviewee, without recording (PF). One 
interview was done through Skype (PP). The average 
duration of the interviews was over 50 min, most of them 
taking place in the humid open lobby of the conference 
resort. Twenty-four interviews were held in English, 
which was not the native language for six of them. Eight 
interviews were in Dutch. 

The interview protocols were slightly different for pro- 
fessionals, new patients and experienced patients. The 
topics were not only derived from the document analysis 
but also from four pilot interviews and the personal 
knowledge of MW and JK and the expertise of TA. The 
topics dealt with: the expected role of patient partici- 
pants, their selection, preparation and support, and the 
expected or provided contribution to the OMERACT 
conference. 'Fatigue' was added as a potential probe 
because publications had already shown that this topic 
deserved special attention with regard to our research 
questions. 22-24 Participants with long-term experience in 
OMERACT were asked retrospectively to describe their 
memories of the discussions and decisions taken about 
patient participation before and after 2002. Their recol- 
lections might be characterised as 'oral history'. 

Selection of interviewees: At OMERACT 10, a total of 172 
delegates participated, 152 professionals and 20 patients. 
Nine patients attended the conference for the first time. 
All interviewees, except for one patient participant from 
the hosting country, were selected by MW and JK follow- 
ing an emergent purposive sampling approach. 21 They 



used a list of attendees provided by the congress agency, 
covering four of five criteria found to be important 
(stakeholder background, gender, geographical spread 
and number of OMERACT conferences attended) . The 
criterion 'opinion about patient involvement' was 
assessed on the basis of authors' insight of the partici- 
pant as being 'positive' (eg, contributing to the patient 
perspective workshop or involving partners in own activ- 
ities), 'indifferent' or 'sceptical' (eg, resistant, not collab- 
orating with partners) . When it became clear during the 
process of data collection that certain criteria were not 
well covered, new participants were approached till 
maximum variation was realised. For example, two inter- 
viewees, who were chosen because of their previously 
reported criticism of involving patients, showed a consid- 
erable change in perception of patient involvement in a 
positive way. For this reason, two more interviews were 
arranged with professionals who had expressed critical 
comments during the last conference. Finally, to ensure 
the opinions of young investigators, two OMERACT 
Fellows were approached, one undertaking a PhD in 
translational research and the other a postdoctoral 
researcher active in clinical research. 

Saturation was defined as a repetition of data; theoret- 
ical saturation as achieving sufficiently robust empirical 
data to support and describe the identified themes and 
main categories. Saturation was discussed and agreed 
within the research team. In total, the perceptions and 
experiences of 16 patient participants and 16 profes- 
sionals were collected (table 1). 

Data analysis 

A thematic content analysis focused in particular on the 
reported contributions attributed to patient participants. 
Coding of the interviews was done separately by MW and 
an independent second coder (MK) who had never 
worked with active patient involvement before. This 
resulted in 211 detailed codes that were then combined 
into 27 subcategories. During several meetings, the project 
team, representing various backgrounds, discussed the 
codes and subcategories from a variety of perspectives and 
sought natural groupings or categories within the data. 
Triangulation was used in two different meanings: first, as 
a means of verifying findings against another source 
(interview) or another method (document analysis) and 
of enhancing the validity of the data. Second, as a means 
of enriching the data collection and improving the face- 
validity by synthesising findings from the document ana- 
lysis with the personal memories and experiences of 
respondents who looked back in time. By doing so, gaps in 
the document analysis could be filled in. 

The relevance and validity of the analysis and inter- 
pretation of the data were increased by the involvement 
of an external expert in qualitative health research (TA) 
as well as by inviting one of the patients (SC) who 
attended OMERACT 10 for the first time to join the 
research team. As a patient research partner, 25 she was 
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Table 1 Characteristics of interviewees 




Patient research 




Professionals 


partners 




Sex(M:F) 12:4 


7:9 


Interview code 



Professional background or 
diagnosis 




Number of OMERACT conferences 
attended 



i 



>5 

Interview in relation to OMERACT 
conference 

Before 

During 

After 

Geographical spread 



Research background 



10 practising rheumatologists 
3 full-time researchers 
3 other professionals 



2 
8 
6 

6 countries 
2 continents 

10 senior researchers 
1 research fellow 

1 postdoctoral researcher 

2 pharma representatives 
2 staff members 



10 rheumatoid arthritis 
2 vasculitis 
2 ankylosing 
spondylitis 
1 fibromyalgia 
1 gout 



3 
1 
4 



1 

16 
5 

7 countries 
4 continents 



PA to PF, PO, PP 
PG to PN 



RA to RG, RJ, RK, RY 
RH 
Rl 

DA, DD 

DB, DC 



OMERACT, Outcome Measures in Rheumatology. 



involved in the coding, data analysis and data interpret- 
ation to guarantee the patient perspective. To protect 
the anonymity of the participants, all quotes are pre- 
sented in the 'she '-form. Quotes of professional 
researchers are indicated by 'R' and those of patient 
research partners (in short: 'partners') by 'P'. 



RESULTS 

History of patient involvement at OMERACT 

OMERACT started in 1992 as an initiative to overcome 
the problem of widespread and inconsistent use of many 
different outcome measures in rheumatoid arthritis 
(RA) clinical trials. The objective was to improve "the 
accuracy and responsiveness to change of clinically rele- 
vant (to patient and clinician) endpoints." 26 
Rheumatologists from many countries met in Maastricht 
and achieved consensus on a core set of outcomes for 
RA. The RA core set was endorsed by WHO. 27 The 
initial stand-alone conference was sufficiently successful 
to be followed by conferences in alternate years, con- 
tinuing the discussion and consensus building about 



new core sets for other rheumatic diseases and new 
measurement instruments. 

During the fifth OMERACT conference (2000), parti- 
cipants discussed the concept of a minimum clinically 
important difference (MCID). Based on methodological 
arguments, a growing interest in PROs emerged, culmin- 
ating in a spontaneous proposal at the final session to 
invite patients to the next conference. All participants 
voted in favour of this proposal. 28 The chair of the con- 
ference felt confident about the proposal because it had 
been discussed previously in the organising committee, 
although no decisions had been taken. Participants of 
the MCID module argued that patient perspectives 
should be explored further 29 and took responsibility for 
identifying 11 patients to join OMERACT 6 and to 
review the RA core set. 

Our document analysis revealed the unconditional 
positive reception of patient delegates at OMERACT 
conferences, and partners confirmed that concerns 
regarding their involvement were misplaced. They felt 
that their reception was extremely welcoming. "There 
was a tangible feeling of relief and a belief that patients' 
views and opinions would be listened to and 
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incorporated into the deliberations." Also, the organi- 
sers were excited and called the patient involvement 'a 
tremendous success'. 31 

Between 2002 and 2012, a total of 57 partners with dif- 
ferent rheumatic diseases have participated as full dele- 
gates with equal voting rights. 32 Their role and 
contributions have developed over time. At the first con- 
ference (2002), they formed a homogeneous group of 
people with RA with little or no experience in scientific 
research. The level of involvement in the conference in 
general was relatively low, support was not organised and 
the number of sessions patients attended was limited. 
Contributions centred on participation in the workshop 
discussions about the severity of fatigue and the defin- 
ition of low disease activity, although there was a keynote 
speech at the opening ceremony. 30 In contrast, by 
OMERACT 11 (2012), the partners were a heteroge- 
neous group with different rheumatic conditions and 
different levels of experience, competences and cultural 
background. They received a preconference information 
pack and were actively supported by a preconference 
dinner, a glossary, training sessions and a buddy system. 
They carried out a variety of tasks similar to profes- 
sionals such as giving plenary presentations, cochairing 
breakout sessions, reporting back from breakout sessions 
and preparing consensus statements. Several partners 
became coauthors of peer-reviewed publications. 

Patient contributions to OMERACT meetings and outcome 
research 

Interviewees reported a variety of contributions made by 
partners during the conference where they are an inte- 
gral part of the deliberative and consensus-building 
process. 18 These examples are presented below and 
compared with the document analyses when 



appropriate. Because research in the domain of fatigue 
has been reported as the most illustrative example, the 
contributions in this area will be described in more 
detail. Using the methodology described above, we iden- 
tified five main categories from the comments made 
during the interviews with OMERACT participants 
(table 2): contributions to the research agenda; devel- 
opment of core sets; development of PROs; culture of 
OMERACT; consequences outside OMERACT. Finally, 
we will highlight some of the challenges that emerged 
from the interviews. 

Contributions to the research agenda 

From the very beginning, partners had a significant 
influence on the research agenda in the field of 
rheumatology by participating in OMERACT workshops 
and small group discussions. They identified new 
outcome domains that are relevant from their perspec- 
tive. 33 The first Patient Perspective Workshop, attended 
by 11 patient participants and 41 professionals, focused 
on the development of "valid outcome instruments that 
incorporate the perspective of the patient and to 
prepare the evidence and arguments for their inclusion 
in the (RA) core set." 24 The preconference paper 
pointed out the methodological and political challenges: 
How to elicit and incorporate preferences of patients in 
RCT's? 32 The workshop had been specifically arranged 
to support the partner contributions including a pre- 
workshop and a postworkshop meeting. The workshop 
identified the subjective experiences of RA not encom- 
passed in the RA core set but having important conse- 
quences of the disease: a sense of well-being, fatigue and 
disturbed sleep. 24 

After the first conference was attended by partners, it 
became apparent that perspectives of professionals and 



Table 2 Main and subcategories from the analysis of patient contributions to OMERACT meetings and outcome research 
since 2002 

Impact of a decade of patient involvement in OMERACT 

Patient-reported Culture of Consequences 

Research agenda Outcome core sets outcomes OMERACT outside OMERACT 



Generating 
challenging ideas 
Identification of 
patient-relevant 
research topics: 

► Well-being 

► Fatigue 

► Sleep disturbance 

► Flares 



Identification of patient 
relevant domains to include 
in core sets for clinical trials: 

► Fibromyalgia 

► Psoriatic Arthritis 

► Vasculitis 

► Gout 

► MRI 

► MCID 

► Remission 



Reality check 



Acceptable, understandable ► 

and feasible outcome ► 

measures for ► 

► Monitoring adverse ► 
events ► 

► Work productivity 

► Flares ► 

► Psychosocial 
interventions 



Attitudes 

Communication 

Perceptions 

Motivation 

Relational 

empowerment 

Personal 

benefits 



► Local initiatives 

► Local and 
national networks 
of partners 

► EULAR 

► COMET 

► ISDM 



OMERACT, Outcome Measures in Rheumatology. 
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patients differed and that more research was needed to 
articulate patient priorities. 34-37 Partners emphasised 
the need for a holistic approach to people with arth- 
ritis. 30 The acknowledgement of the discordance of per- 
spectives initiated new studies looking into the 
preferences, opinions and experiences of people with 
rheumatic diseases and developing patient- 

derived core sets. 40 This made participants more aware 
of the emerging patient perspective: 'the whole realm of 
things we haven't looked at' (RA). New topics emerged: 
remission, pain, flares and foot problems. One inter- 
viewee clearly stated that partners "inspired me for new 
projects to study the variety in new productivity outcome 
measures" (RK). 



Case study of fatigue 

Since 2002, when partners identified new topics for 
research, studies have been initiated with the firm 
involvement of partners in the field of sleep distur- 
bances, flares and well-being. The most progress has 
been made in fatigue, and the emergence of fatigue as a 
relevant outcome measure in RA provides an illustrative 
case history. When asked for the greatest benefit of 
including partners in OMERACT conferences, intervie- 
wees unanimously confirmed that the topic of fatigue 
would not have been on the research agenda without 
partners expressing their concerns about fatigue as an 
often neglected symptom of their disease and without 
the listening of receptive professionals. One of the part- 
ners attending OMERACT 6 recalled 

I can't remember who brought up the subject, but 
someone mentioned fatigue. And that was the occasion 
when one of the other delegates said 'well, everybody 
gets tired'. One patient shot to her feet and said 'no, it's 
not, it's not like anything you've ever experienced; it's 
not tiredness; it's a complete wipe-out'. (PM) 

Early descriptions of fatigue at OMERACT 6 and 7 led 
to substantial qualitative and quantitative research. The 
first studies investigated the prevalence and severity of 
fatigue in RA and how patients describe their 
fatigue. 23 41-43 The next step comprised a systematic 
review of measurement instruments for fatigue 44 that 
explored the rigour of existing measurement tools and 
the need to develop patient-derived instruments that are 
trustworthy, capturing the concepts and language of 
patients. Furthermore, a standardised visual analogue 
scale, opportunities for electronic gathering of data and 
exploring mechanisms of fatigue that could guide 
researchers in the development of effective interven- 
tions, were added to the research agenda. New data pre- 
sented at OMERACT 8 (2006) showed that fatigue is not 
a consequence of RA, but an independent variable that 
adds new information to the existing RA core set. 45 46 
This new perception resulted in the acceptance of 
fatigue as an important outcome for clinical trials. 47 48 



Fatigue was subsequently added to the RA core set as a 
recommended outcome. 47 

More powerful instruments for measuring fatigue in 
RA have since been devised and validated, starting from 
the perspective of the patients 49 50 Outside OMERACT, 
researchers initiated similar studies, focusing on the 
communication between patients and health profes- 
sionals in the consultation room. 51 

The thematic document analysis provided additional 
evidence for the statement that, without patients raising 
their voice at OMERACT 6, fatigue would not have been 
high on the research agenda. The issue of fatigue was 
not new for rheumatologists. 52-54 Fatigue was a symptom 
that was regularly reported during clinical consultations, 
but not incorporated in guidelines for monitoring and 
managing. Fatigue in ankylosing spondylitis was identi- 
fied by physicians and incorporated in a disease status 
questionnaire. 55 During OMERACT 3 (1996), delegates 
carried out a ranking exercise trying to prioritise psycho- 
social measures in musculoskeletal diseases. The discus- 
sion groups identified outcomes such as pain, 
depression, anxiety and fatigue as major concerns. 56 For 
fatigue, eight examples of measurement instruments 
were given. 57 However, after this workshop, nothing hap- 
pened for 6 years, until patients raised the urgency of 
fatigue as a serious symptom. 

Retrospectively, professionals admitted that they had a 
blind spot for fatigue in RA and that only hearing from 
partners at OMERACT made them change their percep- 
tion of fatigue as an important outcome 

Because when I was working in oncology before, during 
university training, of course we saw that the patients were 
lying in bed all day and we knew they were exhausted, call 
that fatigue. But patients with RA, we were ignorant. (RC) 

Another physician involved in OMERACT from the 
start 

We were first discussing on fatigue and to be honest: 
I never ever had before heard of fatigue being a problem 
in rheumatology. So it got into my mind and then I got 
thinking about it and then, when I was back, I asked 
patients if they felt fatigue and I got nearly a 100% posi- 
tive response. So it was like a coming out, you know. I lis- 
tened to the patients before but bringing it to a specific 
topic, that was really what I learned at OMERACT. (RA) 



Development of core outcome measurement 
sets and PROs 

During the first two conferences including patients, the 
focus of partners' contributions was on agenda-setting 
and identifying relevant outcomes for clinical trials. 
Then partners became gradually involved on different 
levels in other OMERACT activities, varying from being 
consulted (eg, in a Delphi process) to full collaboration 
(as a partner and as a coauthor). They contributed by 
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identifying domains that are relevant for disease-specific 
core sets for psoriatic arthritis, 58 fibromyalgia, 59 60 
gout 61 and vasculitis. 62 Furthermore, they contributed to 
the development of core outcome measurement sets for 
methodological or clinical concepts such as MCID and 
remission. 63 Partners have also played a role in the 
assessment of the feasibility of instruments and core sets, 
one of the three key components of the OMERACT 
Filter. 64 

Partners have been helpful in the development of 
PRO measurement instruments in the field of work 
productivity, monitoring adverse events, 65 flares 66 and 
psychosocial interventions such as self-management pro- 
grammes. 67 At the 2010 conference, during the plenary 
session on flares in RA, one of the partners gave a per- 
sonal testimony about the devastating impact of the 
unpredictable nature of RA. A professional in the audi- 
ence was surprised and reported: "It demonstrates that 
the disease activity fluctuates more than we can see in 
our data: our instruments are more flat, and by the 
limited frequency of measuring we filter fluctuations 
out" (RI). 

Regulators require strong evidence for the effective- 
ness of new medicines by demonstrating accurately that 
they reduce structural progression as well as patient 
important outcomes. By doing both of these, developing 
standards for high quality imaging techniques and 
exploring new PROs and translating them into valid and 
feasible measures, OMERACT has been extremely 
advantageous for the negotiations with regulators about 
the registration and relatively generous reimbursement 
of new biological agents 

I think, to be really honest, the patient involvement 
process in OMERACT and the changes in outcome mea- 
surements and the use of them in the drug tests has 
made a real difference for so many patients. (RB) 

Culture of OMERACT 

In spite of the initial unanimous vote to invite patients, 
some researchers were concerned about changing the 
layout of the conference 

My original expectation of a limited contribution was 
based on fear that patients were not able to transcend 
their personal experience and to generalize . . . new stake- 
holders often don't have knowledge about clinometric. 
(RE) 

In retrospect, researchers explained that they deferred 
to the proposal in order to reflect a core principle of 
OMERACT of not immediately rejecting new ideas: "To 
respect and listen rather than just react" (RA). 

Looking back, the number of professional participants 
who were in favour of partners at the conference slowly 
increased: "I was impressed by the very good working 
flow" (RC). Participants confirmed that the presence of 
partners has changed their way of thinking and talking. 



"They made my blind spot visible" (RK) and another 
professional reported 

Now what we have found is, and I changed my view, [be] 
cause it wasn't only from OMERACT. As I got to know 
more and more patients, I realized, this sounds stupid 
because it's so obvious but it wasn't obvious to me, that a 
patient isn't their disease. A patient is a person who 
happens to have a disease. What a big difference. 
Because if you're a person that happens to have a 
disease, then for example you might have incredible skills 
in an area that might be very useful to move a clinical 
trial forward. So once I came to that realization then 
patient involvement becomes an absolutely obvious and 
integral part of moving forward. (RA) 

Partners improved communication and brought 
dynamics to the dialogue because they are motivated 
and constructive, without a personal agenda. At a con- 
ference such as OMERACT, where the discussion about 
methodology may become extremely technical, partners 
reminded participants of the common goal of the con- 
ference by providing a human face of a person living 
with the condition day by day. Their presence made par- 
ticipants more explicit about the objectives of sessions 
and more explanatory about the terms and concepts 
under discussion. Together with a reduced use of 
jargon, this 'forced' simplification resulted in fewer mis- 
understandings for everyone. 

For some professionals, the presence of partners com- 
plicated the communication. Some believed that part- 
ners slow down the process because they are not familiar 
with technical issues. Others felt disinclined to say what 
they wanted to out of respect for partners or hesitated 
to criticise them. One researcher felt embarrassed in the 
presence of partners and put her own expertise aside to 
keep things simple: "Patients didn't sometimes under- 
stand the objective of the research, which hindered us" 
(RK). One of the partners admitted that "it is a thin line 
between providing input and causing irritation" (PN) . 

An analysis of the responses of patients attending 
OMERACT for the first time showed that new partners 
experienced a significant learning curve and a variety of 
personal benefits. Results from this study suggest that in 
fact all participants learnt from the contact with other 
stakeholders. During this process, participants gained 
trust, respect and understanding, reflecting the emer- 
gence of relational empowerment: 

Patients were a kind of sparring partner when I entered 
a relatively new area. That was fun and did clarify a lot. 
(RK) 

Relational empowerment in the context of health 
research can be understood as a process in which trad- 
itional doctor-patient relationships transform into equal 
partnerships enabling mutual learning processes. 68 All 
participants become stronger by sharing knowledge and 
responsibilities, and educating and helping each other. 
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The reported benefits were easiest to identify at the 
beginning when the level of involvement was still low 
They became more diffuse when partners were structur- 
ally involved as full and equal collaborators. One inter- 
viewee mentioned 'a reality check' as an important 
benefit of partners attending the conference. For profes- 
sionals, it offered the opportunity to check the relevance 
of the scope of their research: are we doing the right 
things according to patients and are we using the right 
tools and methods? It is a belief of professionals that this 
kind of feedback is important to legitimise their 
research and, together with the belief of partners that 
without this research no innovations will take place, it 
strengthened the mutual empowerment of both. 

Consequences outside OMERACT 

The lessons learnt at OMERACT were noticed by the 
outside world. Partners returning home after the confer- 
ence have continued to introduce patient participation 
in local and national research projects or established 
networks of patient research partners. 69-72 Some dele- 
gates published a working framework for incorporating 
the patient perspective in outcome research. 73 With the 
input from several OMERACT participants, the 
European League Against Rheumatism developed 
recommendations for the inclusion of patient represen- 
tatives in scientific projects. 25 Following these recom- 
mendations, a new patient reported quality of life 
instrument for RA was developed and validated. 40 74 
Based on the experiences of OMERACT, the organising 
committee of the sixth International Shared Decision 
Making conference decided in 2011 to invite patient 
participants. 75 In the same year, OMERACT delegates, 
partners and professionals participated in the second 
COMET conference, demonstrating how the OMERACT 
methodology can be utilised in other disease areas. 12 

Remaining challenges emerging from the interviews 

The role and contribution of patient participants have 
changed over time and procedures for patient selection 
and support have been developed in order to identify 
patient participants who are able to make a difference. 
There is still a debate going on whether patients should 
be selected through strict criteria such as education, 
communication skills, attitude and familiarity with scien- 
tific research. Some argue that an expert meeting like 
OMERACT needs expert patients who have extended 
knowledge about methodologies of outcome research 
and are able to provide a kind of aggregated patient 
input. At OMERACT, this group represents a minority of 
delegates who are reluctant to allocate the same rights 
and power to partners as to the professionals. The vast 
majority believes that many patients are able to contri- 
bute to an OMERACT conference and emphasise that a 
heterogeneous group of partners in age, gender, condi- 
tion, experience and cultural background are advanta- 
geous for the conference. They intend to develop full 
representative participation in all phases of research by 



including partners in working group activities between 
conferences. Finally, some participants point out the 
potential risks of partners who become too experienced. 
They appreciate the naive input as a patient, with a 
minimum of preparation and reflection. They assume 
that as soon as you start thinking about your contribu- 
tion, you lose the unique, individual perspective and 
become a patient-expert who aligns too easily with 
professionals. 

Professionals shared the opinion that partners need 
training, although they reported different ideas about 
the content and aims of such training. Experienced 
partners as well as novice researchers felt that any new 
participant has to learn the OMERACT objectives, 
culture and procedures first, before they can become 
fully productive, mostly at the second or third confer- 
ence. This accords with the expectations of partners 
who attended OMERACT for the first time. 

Overview of findings 

These results show that a decade of patient involvement 
has been successful and had a significant impact on 
various aspects of outcome research. Perspectives of 
patients are different from those of health professionals. 
A broad consensus exists that partners at OMERACT 
have played a vital role in identifying domains relevant 
from the perspective of patients and in developing new 
PROs such as fatigue, sleep quality, flares and work pro- 
ductivity. We have shown that patient involvement at dif- 
ferent levels and in different phases improves the quality 
of outcome research, especially in the area of fatigue. By 
combining the evidence-based knowledge of researchers 
and the experiential knowledge of patients, a synthesis 
of both kinds of knowledge has been achieved and 
documented. The benefits are assessment tools that 
accurately measure what really matters to patients, are 
formulated in understandable language and are user- 
friendly. Other benefits go beyond improving clinical 
outcome research and include improved communica- 
tion, mutual empowerment, changed attitudes and sub- 
stantial consequences outside OMERACT. 

DISCUSSION 

We set out to describe and evaluate the contribution of 
patients as partners in rheumatology outcome research, 
reviewing their impact on the research agenda and the 
culture and process of the OMERACT conference. 
The document analysis provided the recorded facts 
while the interviews allowed an exploration of the inten- 
tions, attitudes and perceived benefits or harms of 
patient participation that complements the document 
analysis. Since validated methodologies for demonstrat- 
ing the impact of collaboration with patients in the 
context of research are lacking, a responsive interview 
methodology seemed to be a good approach. 

Both the strengths and limitations of this study relate 
to the personal experience of the first and last authors 
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as participants in the developing process of patient- 
partner participation. Having witnessed the OMERACT 
process, actors and concerns of both the patient com- 
munity and the research community was advantageous 
during the development of interview protocols, recruit- 
ment and selection of respondents and data analysis. For 
instance, the knowledge of the opinions of other partici- 
pants made it possible to achieve maximum variation. 
Also, the active involvement in the support and training 
of partners created an adequate awareness of the rele- 
vant items to include in the study. The drawbacks of this 
engagement are the risks of subjectivity, blind spots and 
overidentification or underidentification with particular 
stakeholders. These risks have been addressed by apply- 
ing strict quality measures for scientific rigour in qualita- 
tive, evaluation research. 

The composition of the research team purposely 
included two external experts in qualitative research and 
a patient research partner, who was actively involved in 
the coding of interview transcripts and distilling relevant 
categories for impact, which reduced the risk of subjectiv- 
ity. Bias was avoided by the check-coding procedure in 
the analysis of the transcripts as at least two researchers 
independently coded each transcript, after which the 
whole team discussed the codes until consensus was 
reached. Saturation was also part of the discussion in the 
whole team. The inclusion of various stakeholder per- 
spectives prevented one-sidedness. No signals were identi- 
fied to suggest that interviewees have simply given 
desirable answers, or just been friendly to the interviewer. 
Some interviewees have been rather critical, reporting 
several barriers for structural involvement of patients in 
research, but have always added constructive suggestions 
for improvement. Peer debriefing by an independent col- 
league (TA) further helped to prevent bias. 

Other limitations relate to the difficulties of demon- 
strating the 'impact' of patient involvement. 9 In 
OMERACT, there is a strong belief that patient participa- 
tion works, a belief that is nourished by the worldwide 
transition towards more patient-oriented healthcare and 
health research. 76-79 The assumption, however, that the 
long-term involvement of patients as equal partners 
guarantees sustainable inclusion of the patient perspec- 
tive in outcome research complicates a thorough evalu- 
ation and makes it difficult to distinguish between 
expected, perceived and actual contributions. Many par- 
ticipants, not only partners but also young researchers 
and other newcomers, are not able to identify their own 
contribution and may not see how their input is 
reflected in the final outcomes. Partners reported 
almost unanimously of not being able to confirm sub- 
stantial contributions during their participation, but 
they believed they did. More experienced participants, 
mostly professionals, were less reluctant in reporting 
illustrative examples of patient contributions. 

In a dialogue and consensus-based conference such as 
OMERACT, many (f) actors contribute to the final out- 
comes. A linear causal relation between patient 



involvement and impact is therefore hard to establish; 
the processes of involvement are rather influenced by 
and influencing many (f) actors in a mutually interactive 
way. We found that when the level of involvement of 
partners increased from consultation to collaboration, it 
became harder to solely attribute individual or group 
contributions to the final outcomes. Because neither 
partners nor professionals act as a representative of any 
group or constituency, it remains difficult to determine 
the influence of particular groups or individuals. 
Participation proved to be a dynamic process, especially 
when tasks were equally performed by patients and 
researchers, and when the dialogue between both took 
place not only during the official sessions but also in the 
corridors of the conference. It should be noted that 
striving towards equality is a normative ideal, 80 and fight- 
ing inequalities between patients and professionals is 
and remains an ongoing concern. 'Equality' may be 
seen in two ways: as the formal position of patients at 
the conference (as full delegates, they had the same 
voting rights as professional delegates, received the same 
preconference materials and had access to all confer- 
ence sessions like all other participants) and as equality 
of partnership or collaboration in terms of influence on 
the decision-making process. Given the power inequal- 
ities between patients and professionals, the latter is the 
greater challenge, but nevertheless some of the docu- 
ments and interviews suggest that it has been achieved 
to some extent. Although patients remain only indirectly 
represented in the executive committee (the highest 
decision-making body), our data support the conclusion 
that a small number of experienced patients achieved 
an equal relationship with researchers in their area of 
interest. They obtained the competences that enabled 
them to perform all kinds of tasks at the conference, 
similar to professionals, and provided input that justified 
coauthorship of peer-reviewed articles. We did not 
obtain in-depth information about the question to what 
extent power inequalities between patients and research- 
ers still persist, but we know from the feedback of all 
respondents, including some fellows and researchers 
who attended OMERACT for the first time, that some 
did not feel they were treated equally. To what extent 
this experience was caused by their status of being a 
patient or by the status of a new participant is still 
unknown. 

A last obstacle for demonstrating the influence of 
patient participation is the invisibility of experiential 
knowledge, often hidden in anecdotal stories. It has an 
impact that is rarely claimed by patients and is not per- 
ceived by professionals. Personal comments are normally 
not reported because they are not seen as a valuable and 
valid source of knowledge, 81 and yet clear documentation 
of meetings is required to ensure that patients' contribu- 
tions become visible. 82 Professionals focus on synthesising 
data and may not notice that the dialogue with patients 
works like a reality check, generates new ideas or changes 
their beliefs, behaviour or perception. When partners 



de Wit M, AbmaT, Koelewijn-van Loon M, etal. BMJ Open 201 3;3:e002241 . doi:1 0.1 1 36/bmjopen-201 2-002241 



9 



Impact of patients on OMERACT conferences 



appear to simply agree with the results presented at 
OMERACT, it might look as if they do not have any con- 
tribution to make, but in fact they confirm the value of 
the work under discussion and provide face validity to the 
process. It is for this reason that most professionals appre- 
ciate the feedback and input from partners, although not 
all are aware of this reason. Realising the importance of 
such a reality check is beneficial for the management of 
realistic expectations: do not expect innovative ideas, bril- 
liant suggestions and new concepts when inviting part- 
ners to join research. Their contributions are often more 
subtle and need the attention of a modest and commit- 
ted researcher to be noticed. 

Despite these limitations, we believe that the results 
presented in this study are relevant and valid. It is 
undeniable that there is a growing belief that patient 
involvement has been successful and brought a unique 
added value to the conference. Even those who were ori- 
ginally among the most sceptical participants now report 
that they have changed their perception about the 
expected contribution of patient research partners. This 
study is conducted within the context of a scientific 
research conference in the field of rheumatology, a 
long-term somatic condition. Our ability to generalise 
the findings is therefore limited and extrapolation to 
other research contexts or to other conditions should be 
carried out with care. 
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